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Each country’s education system is intricate and 
unique, underpinned with different policies and 
priorities. International Large Scale Assessments 
(ILSAs) provide valuable insights into how a 
country’s education system is performing. These 
studies gather in-depth contextual information 
about systems, schools, teachers and students; 
and explore the relationships between background 
characteristics and achievement. This information 
can help countries make evidence-informed 
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decisions on aspects of their own education policy 
and classroom practice that they might review and 
adapt to improve learning. 


This briefing provides an overview of England’s 
performance in three of the most recent studies to 
report their findings, considers the experiences of 
other participating countries, and seeks to draw 
lessons for future education policy in England. 


How do the ILSAs differ? 


. nn 5 
England s participation PIRLS and TIMSS are run by the IEA! and aim 


in ILSAs to evaluate students’ mastery of the curriculum, 
i.e. an in-depth understanding of the curriculum 
and how well students are able to apply their 

learning and skills to increasingly more complex 


England’s participation in ILSAs has been 
long-standing. Three of the most well- 


Ue situations. PIRLS and TIMSS are therefore based on 
The Progress in International Reading curriculum content in the participating countries. 
Literacy Study (PIRLS): a study of reading The samples are grade-based (PIRLS year 5, mostly 
at ages 9-10 with a five year cycle. England age 10; TIMSS years 5 & 9, mostly age 10 and 14). 
Peller eiiere| adil lee eich in 2010 PISA is run by the OECD? and aims to evaluate 

The Trends in International Mathematics the education system, with a strong emphasis 

and Science Study (TIMSS): a study of on socio-economic factors and equity issues. 


The assessment aims to measure the application 
of knowledge to real-life situations, and the 
preparedness of young people for society, further 
study and the workplace. Although reading, 
science and maths are always included, each cycle 
The Programme for International Student has one subject as a major focus. The main focus 
Assessment (PISA): a study of reading, of PISA 2015 was science, and in 2018 it will be 
science and maths at ages 15-16 with a three reading. The sample is age-based (15-16 year-olds, 
year cycle. The first cycle was in 2000. mostly in year 11). 

England have participated since 2000 but 

their results are only available from 2006. 


maths and science at ages 9-10 and 13-14 
with a four year cycle. The first cycle was 
in 1995, which included the participation of 
England. 


How did England perform in the most recent ILSAs? 


Born 
The average 
reading score 
of students 
in England was significantly 
higher than in PIRLS 2006 
and 2011, and higher than the (~) 
majority of other countries. 
Started a 
Reception The improvements were 
mainly among boys and 
lower-performing students. 
Only the Russian Federation, Singapore, Hong 
an Kong, Ireland, Finland, Poland, and Northern 
iii ae Ireland significantly outperformed England. 


1 International Association for the Evaluation of Educational Achievement 
2 Organisation for Economic Co-operation and Development 


* This is not exactly the same sample as in 2015, but students from this age group have been sampled in PIRLS 2011, TIMSS 2011 
and TIMSS 2015. 


The average scores of students 
in England for maths have shown 
steady improvement since 1995 
at both year 5 and year 9. 


Average 
scores for 
science 
have 
increased at year 
5 but not at year 9. 


England’s year 5 and year 9 average 
maths scores were higher than the 
international mean and the majority 
of other participating countries. 


England’s average science scores were 
significantly higher than most other countries, 
with only five countries significantly out- ll 


performing England at year 9. 


Singapore, Hong Kong, South Korea, Taiwan, 
Japan and the Russian Federation performed 
well across both subjects and both year groups. 
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“ela “ela in science and at the 


Average science, maths and reading 
scores have not significantly changed 
in England since PISA 2006. 


maths. 


For the first time, 
England was 
significantly above 
the OECD average 
in reading, mainly because 
of decreases in other OECD 
countries. 


The top performers in science 
were Singapore, Japan, Estonia, 
Taiwan, Finland, Macao, Canada, 
Vietnam and Hong Kong. 


England performed significantly better in science than many of 
the Scandinavian countries (Norway, Sweden, Denmark), and 
several other industrialised countries (eg France, Spain, USA). 


OECD average for 
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Started 9Q—— 


Reception 


2015 
Participated 
in PISA 


England has participated in other ILSAs, such as OECD’s Survey of Adult skills CPIAAC) in 2011 
and 2012, as well as IEA’s International Civic and Citizenship Education Study (ICCS) in 2009. This 


document focuses on the most recent ILSAs: PIRLS 2016, TIMSS 2015 and PISA 2015 and considers 
them alongside some key policy areas in England. 


How is England performing over time? 


There are encouraging improvements 
in achievement at primary level, but 


less SO among secondary students 


where performance has not changed 


significantly. 


Year 9 Year 5 


Year 10/11 


England performed well internationally in science and 
reading at primary level. Performance is weaker in 
maths, with a larger gap between achievement of top 
performing countries and England, however TIMSS 
does show gradual improvements. 
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Please note: The top-performing country varies between studies and across cycles, but includes Chinese Taipai, Finland, Hong Kong, Korea, Russian 
Federation, Singapore, Shanghai China and Sweden. 
The PIRLS, TIMSS and PISA achievement scales are defined with a centrepoint set at 500 and a standard deviation of 100. However, each study’s scale 
is determined independently during its inception, or in PISA’s case, the first time that subject was a major focus. Direct comparison of average scores 

across studies and across subjects within studies should therefore not be made. 


Across all studies the ‘Asian tigers’ dominate 

as top-performing. The educational success of 
these East Asian countries is ascribed by some to 
the high value placed on education and a belief 
that effort rather than innate ability is the key to 
educational achievement (Boylan, 2016). While 
undoubtedly a factor, these cultural attitudes 
alone cannot explain the rapid development 

in education made by these countries, since 
these traditional values are not of recent origin. 
Another thing they have in common is reforms in 
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recent decades based on theories of learning and 
competency-based education, as typified by the 
Singapore slogan Jeach /ess, learn more (Cheng, 
2017). They also share high status of teachers 
and opportunities for continued professional 
development. However, there should be some 
caution in seeking lessons from these countries 
since they are beginning to have concerns about 
levels of stress and unhappiness among their 
children and young people caused by pressure for 
academic success. 
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Insights for policy 


Examining England’s performance in at least four 
cycles of each ILSA provides a useful overview of 
education in England. The studies are particularly 
valuable when they are used to investigate 
specific policy issues, such as gender differences, 
attainment in Science, Technology, Engineering 
and Maths (STEM) subjects, the impact of socio- 
economic status and school climate. 


ILSA findings in England reflect international 
patterns in relation to gender differences in maths. 
The average maths score for boys was higher than 
that of girls in both TIMSS Year 5 and PISA. This 
difference was also found in 2017 GCSE maths 
results, where 16.5 per cent of boys were awarded 
at least an A or a 7 grade, compared with 14.7 per 
cent of girls. 


Around half of countries had a gender gap in 
favour of boys in TIMSS (Y5). In PISA the male 
advantage in England was larger than the OECD 
average, although in general boys outperformed 
girls internationally. 


This gender gap was not present in science 
performance in PISA or in either of the year groups 
in TIMSS. This is an encouraging sign since boys 
have outperformed girls in earlier PISA studies and 
a gender gap is found in many other countries. 


In reading, in common with the great majority of 
other countries, girls outperformed boys in both 
PIRLS and PISA. The gender gap was similar to 
international averages in both studies. The gap has 
narrowed in PIRLS, due to boys improving their 
average performance. 
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Girls lack confidence and self-belief 


“.. the strong relationship among self-beliefs, gender and performance in mathematics and science 
hints that countries may be unable to develop a sufficient number of individuals with strong 
mathematics and science skills partly because of girls’ lack of confidence in their abilities.” 


OECD (2015) 
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England’s young people have the potential to compete 
internationally in science-related fields 


As the government seeks to enhance the nation’s 
competiveness on the world stage it will be 
increasingly important to ensure all our young 
people are equipped with STEM skills and make 
the most of their talents. With two of the ILSAs 
collecting information on maths and science, 
England’s participation can provide some insight 
into the state-of-play so far. 


¢ TIMSS and PISA results show that England is in 
a good position to compete internationally in 
science-related fields, but lags behind in maths. 


¢ The majority of students in PISA view school 
science as relevant to their future and 28 per 
cent aspire to a career in a STEM field, higher 
than the OECD average of 24 per cent. This has 
increased from PISA 2006 when only 16 per cent 
aspired to a STEM-related career. 


¢ The most popular STEM-related aspirations were 
‘engineer’ (six per cent) and ‘medical doctor’ 
(five per cent). 


“Weaknesses in the Uk’s skills base have contributed to its long-standing productivity gap with 
France, Germany and the United States. It performs poorly on intermediate professional and 
technical skills, and is forecast to fall from 22nd to 28th out of 33 Organization for Economic Co- 
operation and Development (OECD) countries for intermediate skills by 2020.” 


GB. Parliament. HoC (2016) 


% of students indicating a 
career aspiration in this field 


. boys girls 
England 


One-in-five girls (21 per cent) in England are 
interested in a career as a health professional, 
compared to seven per cent of boys (Jerrim 

° and Shure, 2016). Conversely, boys in England 
: are more likely to aspire to become a scientist/ 
> engineer than girls (16 and 6 per cent 
respectively). 


This gender difference is not unique to 
England. Internationally, boys are more than 


Boys and girls have different career aspirations 


ICT 
ie professional 


i Health s 
professional 


Es Scientist/ 


engineer 


boys girls : 
OECD : 


twice as likely as girls to expect to work as 
engineers, scientists or architects (OECD, 

2017a). Additionally, nearly five per cent of 

boys expect to work as ICT professionals, on 
average, across OECD countries. However, 

less than one percent of girls expected : 
the same. Girls are almost three times : 
more likely than boys to expect to work as 
doctors, veterinarians, nurses or other health : 
professionals. : 
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Socio-economic gap 


The socio-economic gap is relatively high in England, but many 
students also overcome disadvantage 


Social mobility is a key priority for the Government, ¢ In PIRLS and TIMSS, the number of books in the 


which has an ambition to close the attainment gap. 
ILSAs can help point out in more detail the extent 
to which England provides opportunities for young 


home is used as a measure of socio-economic 
status*. In PIRLS, there is a difference of more than 
100 points between students reporting having the 


people from all socio-economic groups. most and least books at home. In TIMSS 2015, at 
both year 5 and year 9, there are similarly large 


differences. 


“_. disadvantaged children remain - on * In PISA, the main measure of socio-economic 
average - four months behind at the end of status is the Economic, Social and Cultural Status 
reception year, 11 months behind at the end (ESCS) index. This is based on parental occupation 
and home possessions. The impact of socio- 
economic background in PISA 2015 is similar to 
that of many other countries. 


of primary school and 19 months behind at 
key stage 4” 


GB. Parliament. HoC. (2017) 


¢ In England, the relationship between ESCS and 
average maths score has not improved since 2006, 
with the difference between the most and least 
disadvantaged students being equivalent to over 
three years of schooling (Wheater et a/., 2016). 


The socio-economic gap in PIRLS and TIMSS 
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Difference in the average score for students with 
‘less than 10’ and ‘more than 200’ books in the home 


3 See NFER’s research for a more sophisticated socio-economic measure, based on PISA’s index of ESCS, which was used to analyse 
Northern Ireland’s TIMSS data (Bradshaw et a/., 2018). This is not currently replicable with England data, as England does not 
administer a parent questionnaire. 
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The socio-economic gap in PISA science is similar to other OECD countries 


England —————— 


OECD average rey 


400 450 500 550 600 


Range in average scores for students in the 
lowest 25% and the highest 25% by socio-economic status 


Resilience in PISA‘4 


As well as the size of the effect of socio- 
economic status on attainment (i.e. the 
difference in attainment between a typical 
high and low socio-economic status 
student), the strength of the effect is also 
important (i.e. how much variation there is, 
and hence the likelihood that some students 
‘buck the trend’). One way to consider the 
strength of the effect is by examining the 
concept of resilience. 


The term ‘resilience’ refers to the positive 
adjustment that enables individuals 

to overcome adversity; and academic 
resilience can be used to characterise 
students who succeed in school despite 
coming from a socio-economically 
disadvantaged background (Agasisti, 2018). 


Factors which are associated with resilience: 
* a positive classroom disciplinary climate 

* availability of extra-curricular activities 

¢ low turnover of teachers. 

Encouragingly, around a third of students 
in England are classed as ‘resilient’, 
overcoming a disadvantaged socio- 
economic background to achieve top 


scores, compared with an OECD average 
of 29 per cent. 


4 See NFER’s research (Wheater et a/., 2016) for 
further findings on resilience in PISA 


For more information visit www.nfer.ac.uk/international 


Germany is reducing the 
socio-economic gap 


In PISA 2000, Germany had one of the 
lowest levels of social equity amongst all of 
the OECD countries, which caused major 
shock to the education system. This swiftly 
led to the implementation of large-scale 
reforms including: 


* The 2004 development of a ‘common 
standard’ within subjects across 
Germany’s 16 states. 


The 2006 adoption of a comprehensive 
strategy for educational monitoring, 
including a cross-state comparative study 
(Vergleichsarbeiten) for third and eighth 
grade students. Results of which are used 
for monitoring instead of accountability; 
teachers can see the strengths and 
weakness of their students, allowing 
them to target their most disadvantaged 
students. 


A reduction in the amount of selectivity 
within the education system. 


An increased investment in 
comprehensive schools where children of 
all abilities are taught together. 


The recent PISA results suggest Germany 
has managed to improve in all three 
subjects whilst decreasing inequality in its 
school system. The proportion of resilient 
students has also risen from 25 per cent in 
2006 to 32 per cent in 2015. 


Teachers and headteachers in England are more positive about 
school climate than their students 


PIRLS, TIMSS and PISA include questionnaires 


completed by the headteachers and teachers “The key task for a school leader is to create 
of the sampled students, as well as the a culture - usefully defined as ‘the way we 
students themselves. Headteacher and teacher do things around here’- that is understood 
questionnaires contain questions addressing and subscribed to by the whole school 

the school climate, formulating scales such as community.” 

schools’ emphasis on academic success, teachers’ 


job satisfaction, and problems facing teachers. Bennett (2017) 
These questionnaires also address school safety 
by looking at discipline problems, bullying, and 
how safe and orderly schools are. These findings 
provide contextual information, giving insight into 
what is actually happening in schools. 


¢ In PIRLS, teachers report that their schools are 
safe and orderly and headteachers report that 
issues with discipline are rare more, on average, 
than in other countries. However, the percentage 
of students reporting bullying is higher than in 
many other countries. 


« In TIMSS, headteachers and teachers report 
fewer problems with school climate than in other 
countries while students report bullying at a 
similar rate to students in other countries. 


¢ In PISA, 93 per cent of headteachers report 
that they pay attention to disruptive behaviour, 
compared to an OECD average of 82 per cent. 
However, there is more frequent student- 
reported disruption in science classrooms in 
England than in the average high-performing 
country - especially compared with East 
Asia - although this is similar to many other 
industrialised countries. 


South Korea: the trade-off between high achievement and 
student wellbeing 


South Korea is one of the highest-performing countries in the world. Well-disciplined classrooms 
and long hours of studying are commonly considered to be contributory factors in this success. 
However, it has also consistently scored lowest in various studies when it comes to student 
wellbeing, happiness at school and the general quality of life of young people. In PISA 2015, for 
example, only 53 per cent of students reported that they are satisfied or very satisfied with life, 
compared with an OECD average of 71 per cent. In response to the alarming rate of unhappiness 


and high suicide rates among young people, several reforms have been introduced, including: 


* The 2015 Revised National Curriculum which aims to nurture creative and integrative learners. 


* The ‘Free Semester’ system in middle schools that aims to enhance the happiness and wellbeing 
of students by giving them opportunities to explore non-academic dreams and aptitudes. 


¢ A ‘character education plan’ which encourages schools to implement experience and project- 
based activities, including sports activities and music. 
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England has a high level of teacher shortages compared with other 
countries? 


¢ In TIMSS, headteachers reported more teacher 


“A variety of factors have contributed to 

the growing sense of crisis for schools in 
England struggling to retain and develop 
their teachers. Particularly worrying is that 
the number of secondary school teachers has 


been falling since 2010 and more teachers 
have been leaving the profession for reasons 


recruitment problems than many other countries. 
Around half of year 9 students were in schools 
with shortages in both subjects; 67 per cent of 
headteachers find year 9 science vacancies hard 
to fill. 


In PISA, the number of headteachers reporting 
teacher shortages was high compared with other 


other than retirement since 2012.” countries (England 45 per cent, international 
average 30 per cent). Only two high-performing 
countries complained of more teacher shortages 
than England (Japan 55 per cent, China 64 per 


cent). 


GB. Parliament. HoC. Committee of Public 
Accounts (2018) 
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How high-performing countries develop and support teachers 


“In countries that performed above the OECD average in science, at least 80 per cent of the 
students are in schools that invite specialists to conduct teacher training or organise in-service 
workshops for teachers or where teachers cooperate with each other. This is higher, on average, 


than what is observed among other countries.” 


OECD (2017b) 


Key considerations when looking at ILSAs 


The anticipation that surrounds these ILSAs, especially on the day the findings are released, 
creates an opportunity for misrepresentation of the data. Here are some important notes regarding 
interpretations of findings: 


1. Simply looking at whether a score is higher or lower than in previous study cycles does not 
tell us accurately whether achievement has improved, is stable or is in decline. It is crucial to 
consider whether a score is statistically significantly different - in other words, the differences 
have not arisen solely by chance. 


2. It is tempting to focus on ‘rankings’ when comparing achievement between countries. However, 
‘rankings’ can be misleading because small difference in scores may not be statistically 
significant. 


3. ‘Rankings’ can also be volatile, varying according to the number and mix of countries 
participating in any given cycle. One country’s position relative to other countries is not just 
based on its own performance. It’s also influenced by how many countries participate, as well as 
which countries they are and whether their performance has changed since the last round of the 
study. 


4. No ILSA covers all subjects. Single assessments cannot measure everything, so it is important 
to remember that there are other areas children and young people need to learn, and avoid 
narrowing their curricula. 


5. The data from these ILSAs alone do not provide a magic formula for policy changes that will 
improve achievement. It is therefore important to consider these studies alongside other sources 
of evidence. 
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England is strongest in science and 
primary reading. The weakest subject is 
maths but TIMSS shows some encouraging 
improvements at primary level. 


An encouragingly high number 
of 15-year-olds have ambitions 
to work in STEM-related fields, 
although there are gender 
differences in the type of career 
they aspire to. 


Teacher shortages, 
especially at secondary 
level, are more of a 
problem in England than 
in many other countries. 


Some countries, such as 
Canada, have high scores and 
narrow gender and socio- 
economic gaps. Improvement in 
England’s average score as well 
as a reduction in the size of the 


What are the main 

lessons for England 

from PIRLS, TIMSS 
and PISA? 


Achievement is improving 

at primary level, but not at 
secondary. Attention is needed 
to the transition to secondary 
level to ensure that the 
progress made at primary 
level is maintained. 


Teachers and 
headteachers are, on 

the whole, more positive 
about school climate than 
their students. 


The effects of socio-economic 
status are relatively high, 
although some high-performing 
countries have an even larger 
gap than England. Nevertheless 
there is still a long tail of 


gaps is possible. 


The ‘Asian tiger’ countries dominate in all subjects 
and all age groups. These countries highly value 
their education system and their teachers. They 
collectively place emphasis on effort rather than 

innate ability and have adopted a learning and 
competency-based education. However, there are 
concerns about the impact this ethos may have on 
their children and young people. 


underachievement in England 
which needs to be addressed. 
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